1. Field of the Invention
The present invention relates, in general, to testing computer hardware such as part of a quality assurance process, and, more particularly, to a method and corresponding system for testing and analyzing a computer device or network of devices during running of one or more software applications by simulating loads, e.g., memory, communication, and processing loads, created by applications intended to be run on the tested device or network or devices.
2. Relevant Background
Within the computer industry, an ongoing problem is providing adequate testing and quality assurance of computer devices, systems, and networks prior to initial use and installation by customers. As with other consumer products, it is typically desirable to run a new product or product configuration through a battery of tests to insure that the product performs as expected without performance degradation or without serious problems or errors. With new computer devices, this can be a relatively simple task when testing a standalone computing device such as a personal computer, but testing and quality assurance tasks quickly become very complicated and difficult when applied to large computer devices or systems of networked devices.
Customers in the computer industry often run very sophisticated and large suites of applications to perform their data processing and communicating operations. For example, in the telecommunications industry, telephony customers or businesses that manage the infrastructure for a cellular network have applications that control and manage large numbers of cellular or wireless calls on an ongoing basis. Telephony switching and/or call processing applications generate huge processing, memory, and communication loads within a computer system that far exceeds relational database applications (e.g., another example of applications that can place large loads on computer systems). During operation of such large applications, huge amounts of binary text segments that represent compiled code are executed, memory is accessed, and both inter and intra process communications take place between systems on a particular computing network. Developers and manufacturers often create computer devices and systems to meet the demands of particular customers and as a result, there is a need for being able to test the newly created computer devices and systems under the load or operating conditions for which they are being designed, i.e., under the load conditions their customers will place the machines by running their particular applications and processing their business-specific data.
One useful technique for testing a computer system or network of computer devices is to install the system or network of devices on the customer's premises, such as in an onsite customer-controlled laboratory. In such a setting, the customer makes available their programs or applications and the system or network of devices can be configured to process customer data to provide an accurate and realistic test under the operating conditions intended for the computer system. In this manner, the application(s) or program(s) specific characteristics, such as spatial and execution characteristics, can be tested during operation of the computer system to determine how these effect performance. For example, theoretical performance gains for new computer systems or devices are often much higher than what is actually experienced when the new computer systems or devices are installed on a customer's site to run their applications, e.g., increases in processor clock speed that would tend to indicate a particular increase in performance may not necessarily provide such an improvement due to the particular spatial and execution characteristics of the customer's applications.
Testing of a computer system or network of computer devices can be accomplished effectively with the assistance of the customer, which may include installing equipment on their facilities. However, such testing is often difficult to arrange and schedule and can quickly become a bottleneck when testing indicates problems that may need iterative testing to correct problems. To date, the computer industry has not been able to provide adequate quality assurance and testing of a computer system or its configuration without having the customer's application itself and, in some cases, the customer's data flow. Attempts to run the actual application have not proven successful or have been determined too costly as it is very difficult and time consuming to write programs of the size required to emulate complex customer software suites, e.g., software applications that include or generate millions of lines of code that result in binary text segments that are proportionally large to the amount of compiled code.
Hence, there remains a need for a method and system for testing computer devices and systems under conditions, e.g., spatial and execution characteristics, similar to those created by a particular set of applications or programs to be implemented by an intended customer or end user. Preferably, such a method and system would enable the computer industry to design and build machines to run particular applications before an application(s) is available or without requiring use of the actual application(s).